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Abstract 

Sigma-Delta modulation is a popular method for analog-to-digital conversion of 
bandlimited signals that employs coarse quantization coupled with oversampling. The 
standard mathematical model for the error analysis of the method measures the per- 
formance of a given scheme by the rate at which the associated reconstruction error 
decays as a function of the oversampling ratio A. It was recently shown that exponen- 
tial accuracy of the form 0(2~ rX ) can be achieved by appropriate one-bit Sigma-Delta 
modulation schemes. By general information-entropy arguments r must be less than 
1. The current best known value for r is approximately 0.088. The schemes that were 
designed to achieve this accuracy employ the "greedy" quantization rule coupled with 
feedback filters that fall into a class we call "minimally supported" . In this paper, we 
study the minimization problem that corresponds to optimizing the error decay rate 
for this class of feedback filters. We solve a relaxed version of this problem exactly 
and provide explicit asymptotics of the solutions. From these relaxed solutions, we 
find asymptotically optimal solutions of the original problem, which improve the best 
known exponential error decay rate to r ~ 0.102. Our method draws from the theory 
of orthogonal polynomials; in particular, it relates the optimal filters to the zero sets 
of Chebyshev polynomials of the second kind. 



1 Introduction 

Conventional Analog-to-Digital (A/D) conversion systems consist of two basic steps: sam- 
pling and quantization. Sampling is the process of replacing the input function (x(t))teR by a 
sequence of its sample values (x(t n )) ne z- The sampling instances (t n ) are typically uniform, 
i.e., t n = nr for some r > 0. It is well known that this process incurs no loss of information 
if the signal is bandlimited and r is sufficiently small. More precisely, if supp x C [— 
then it suffices to pick r < r crit := and the sampling theorem provides a recipe for perfect 
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reconstruction via the formula 



x 



(*) = r^x{nr)p{t - nr), 



(1) 



where ip is any sufficiently localized function such that 




(2) 



which we shall refer to as the admissibility condition for (p. Here, x and (p denote the 
Fourier transform of x and (p, respectively. In this paper, we shall work with the following 
normalization of the Fourier transform on R: 



The value p := 1/r is called the sampling rate, and p cr i t := l/r^t = 2f2 is called the critical 
(or Nyquist) sampling rate. The overs ampling ratio is defined as 



We shall assume in the rest of the paper that the value of f2 is arbitrary but fixed. 

The next step, quantization, is the process of discretization of the amplitude, which 
involves replacing each x(t n ) by another value q n suitably chosen from a fixed, typically 
finite set A (the alphabet). The resulting sequence (q n ) forms the raw digital representation 
of the analog signal x. 

The standard approach to recover an approximation x to the original analog signal x is 
to use the reconstruction formula (TXT) with x(nr) replaced by q n , which produces an error 
signal e := x — x given by 



The quality of this approximation can then be measured by a variety of functional norms on 
the error signal. In this paper we will use the norm c roo which is standard and arguably 
the most meaningful. 

Traditionally, A/D converters have been classified into two main families: Nyquist-rate 
converters (A ~ 1) and oversampling converters (A ^> 1). When r = r cr it, the set of 
functions {ip(- — nr) : n G Z} forms an orthogonal system. Therefore, a Nyquist-rate 
converter necessarily has to keep \x(nr) — q n \ small for each n in order to achieve small 
overall reconstruction error; this requires that the alphabet A forms a fine net in the range 
of the signal. On the other hand, oversampling converters are not bound by this requirement 
because as r decreases (i.e., A increases), the kernel of the operator 




A : 



P 



(3) 



Pcrit 




(4) 




(5) 
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gets "bigger" in a certain sense, and small error can be achieved even with very coarse alpha- 
bets A. The extreme case is a one-bit quantization scheme, which uses A = { — 1, +1}- For 
the circuit engineer, coarse alphabets mean low-cost analog hardware because increasing the 
sampling rate is cheaper than refining the quantization. For this reason, oversampling data 
converters, in particular, Sigma-Delta (SA) modulators (see section 2.1 below) have become 
more popular than Nyquist-rate converters for low to medium-bandwidth signal classes, such 
as audio [12]. On the other hand, the mathematics of oversampled A/D conversion is highly 
challenging as the selection problem of (q n ) shifts from being local (i.e., the g n 's are chosen 
independently for each n) to a more global one; a quantized assignment q n G A should be 
computed based not only on the current sample value x(t n ), but also taking into account 
sample values x{%) and assignments qj. in neighboring positions. Many "online" quantization 
applications, such as A/D conversion of audio signals, require causality, i.e., only quantities 
that depend on prior instances of time can be utilized. Other applications, such as digital 
halftoning, may not be strictly bound by the same kind of causality restrictions although 
it is still useful to process samples in some preset order. In both situations, the amount of 
memory that can be employed in the quantization algorithm is one of the limiting factors 
determining the performance of the algorithm. 

This paper is concerned with the approximation theory of oversampled, coarse quanti- 
zation, in particular, one-bit quantization of bandlimited functions. Despite the vast engi- 
neering literature on the subject (e.g., see p2]), and a recent series of more mathematically 
oriented papers (e.g., [5j [151 13 IS ID 12]), the fundamental question of how to carry out 
optimal quantization remains open. After the pioneering work of Daubechies and DeVore on 
the mathematical analysis and design of £ A modulators [5] , more recent work showed that 
exponential accuracy in the oversampling ratio A can be achieved by appropriate one-bit SA 
modulation schemes [7J. The best achievable error decay rate for these schemes was 0(2~ rA ) 
with r « 0.076 in [TJ. Later, with a modification, this rate was improved to r w 0.088 in [TT] . 
It is known that any one-bit quantization scheme has to obey r < 1, whereas it is not known 
if this upper bound is tight [U [7J . This paper improves the best achievable rate further to 
r rj 0.102 by designing an optimal family of Sigma-Delta modulation schemes within the 
class of so-called "minimally supported" recursion filters. These schemes were introduced in 
[7J together with a number of accompanying open problems. The main results of this paper 



see Theorems 4.1, 5.5, 5.6) are based on the solution of one of these problems, namely, 



the optimization of the minimally supported recursion filters that are used in conjunction 
with the greedy rule of quantization. One of our main results is that the supports of these 
optimal filters are given by suitably scaled zero sets of Chebyshev polynomials of the second 
kind, and we use this result to derive our improved exponent for the error bound. 

The paper is organized as follows. In Section [2j we review the basic mathematical theory 
of EA modulation as well as the constructions and methods of [7J which will be relevant to 
this paper, such as the family of minimally supported recursion filters, the greedy quanti- 
zation rule, and fundamental aspects of the error analysis leading to exponentially decaying 
error bounds. As we explain, the discrete optimization problem introduced in [7J plays a 
key role in the analysis. In Section [3j we introduce a relaxed version of the optimization 
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problem, which is analytically tractable. This relaxed problem is solved in Section [4] and 
analyzed asymptotically in Section [5] as the order of the reconstruction filter goes to infinity. 
Section [5] also contains the construction of asymptotically optimal solutions to the discrete 
optimization problem, which yields the exponential error decay rate mentioned above. Fi- 
nally, we extend our results to multi-level quantization alphabets in Section [6j We also 
collect, separately in the Appendix, the properties and identities for Chebyshev polynomials 
which are used in the proofs of our results. 

2 Background on EA modulation 

2.1 Noise shaping and feedback quantization 

EA modulation is a generic name for a family of recursive quantization algorithms that 
utilize the concept of "noise shaping". (The origin of the terminology EA, or alternatively 
AS, goes back to the patent application of Inose et al [10J and refers to the presence of 
certain circuit components at the level of A/D circuit implementation; see also [121 113]-) 
Let (y n ) be a general sequence to be quantized (for example, y n = x(nr)), (q n ) denote the 
quantized representation of this sequence, and (z/ n ) be the quantization error sequence (i.e., 
the "noise"), defined by v := y — q. Noise shaping is a quantization strategy whose objective 
is to arrange for the quantization noise v to fall outside the frequency band of interest, which, 
in our case is the low-frequency band. Note that the effective error we are interested in is 
e = T!f(u), hence one would like v to be close to the kernel of T!f. It is useful to think of 
T^{y) as a generalized convolution of the sequence v and the function ip (sampled at the 
scale r). Let us introduce the notation 

v® T p> :=T*(y). (6) 

Note that (a * b) ® T ip = a ® T {b ® T <p) and a ® T (<p * if)) = (a ® T <p) * if), etc. , where a and b are 
sequences on Z, <p and ip are functions on R, and * denotes the usual convolution operation 
(on Z and K). By taking the Fourier transform of ^ one sees that the kernel of Tf consists 
of sequences v that are spectrally disjoint from <p, i.e., "high-pass" sequences, since <p is a 
"low-pass" function, as apparent from Thus, arranging for v to be (close to) a high-pass 
sequence is the primary objective of EA modulation. 

High-pass sequences have the property that their Fourier transforms vanish at zero fre- 
quency (and possibly in a neighborhood). For a finite high-pass sequence s, the Fourier 
transform s(£) := Y2 s n e2mn ^ has a factor (1 — e 2m ^) m for some positive integer m, which 
means that s = A m w for some finite sequence w. Here, A denotes the finite difference 
operator defined by 

(Aiu) n := w n -tw„_i. (7) 

The quantization error sequence v = y — q, however, need not be finitely supported, and 
therefore v need not be defined as a function (since v is bounded at best). Nevertheless, this 
spectral factorization can be used to model more general high-pass sequences. Indeed, a EA 
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modulation scheme of order m utilizes the difference equation 



y-q = A m u (8) 

to be satisfied for each input y and its quantization q, for an appropriate auxiliary sequence 
u (called the state sequence). This explicit factorization of the quantization error is useful 
if u is a bounded sequence, as will be explained in more detail in the next subsection. 

In practice, p]) is used as part of a quantization algorithm. That is, given any sequence 
(y n ) n >o, its quantization (q n ) n >o is generated by a recursive algorithm that satisfies p]). This 
is achieved via an associated "quantization rule" 

q n = Q(u n -i, U n _2, • • • , y n , • • • ), (9) 
together with the "update rule" 

U n = V(-l)^ 1 ( ) U n-k + y n ~ q n , (10) 



which is a restatement of ([8]). Typically one employs the initial conditions u n = for n < 0. 

In the electrical engineering literature, such a recursive procedure for quantization is 
called "feedback quantization" due to the role q n plays as a (nonlinear) feedback term via 
p| for the difference equation (fsb . Note that if y and q are given unrelated sequences and 
u is determined from y — q via pi), then u would typically be unbounded for m > 1. Hence 
the role of the quantization rule Q is to tie q to y in such a way as to control u. 

A EA modulator may also arise from a more general difference equation of the form 

y — q = H * v (11) 
where if is a causal sequence (i.e., H n = for n < 0) in i 1 with Hq = 1. li H = A m g with 



g E i 1 , then any (bounded) solution v of (11) gives rise to a (bounded) solution u of (I8J) via 



u = g * v. Thus (11) can be rewritten in the canonical form (pb by a change of variables. 



Nevertheless, there are significant advantages to working directly with representations of the 



form (11), as explained in Section 2.3 below 



2.2 Basic error estimates 

Basic error analysis of SA modulation only relies on the boundedness of solutions u of ([8 
for given y and q, and not on the specifics of the quantization rule Q that was employed 



8) for 
) for 



It is useful, however, to consider arbitrary quantizer maps M : y t- >■ q that satisfy ( 
some m and u, where u may or may not be bounded. Note that if u is a solution to (8 
a given triple (y,q,m), then u := Au is a solution for the triple (y, q, m— 1). Hence a given 
map M can be treated as a EA modulator of different orders. Formally, we will refer to the 
pair (M, m) EA modulator of order m. 

As indicated above, in order for a E A modulator (M, m) to be useful, there must be a 
bounded solution it to pi). (Note that, up to an additive constant, there can be at most one 
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such bounded solution once m > 0.) Moreover, one would like this to be the case for all 
input sequences y in a given class V, such as ^ M = {y : < yu} for some \i > 0. In this 

case, we say that (M, m) is stable for the input class V. Clearly, if (M, m) is stable for y, 
then (M, m— 1) is stable for y as well. To any quantizer map M and a class of inputs ^, we 
assign its maximal order m*(M, V) via 

m*(M,V) := sup{m : Vy G V, 3m G such that y — M(y) = A m u}. (12) 

Note that both and oo are admissible values for m*(M, ^). With this notation, (M,m) is 
stable for the class ^ if and only if m < m*(M, ^). 

Stability is a crucial property. Indeed, it was shown in [3] that a stable m-th order scheme 
with bounded solution u results in the error bound 

||e|U°c < IHI^H^Hiir" 1 , (13) 



where ip( m ' denotes the mth order derivative of (p. The proof of (13) employs repeated 
summation by parts, giving rise to the commutation relation 

(A m u)® T <p = u® T (A», (14) 

where A r is the finite difference operator at scale r defined by (A T ip)(-) := </>(•) — ip(- — r). 



The left hand side of (14) is simply equal to the error signal e by definition. On the other 



hand, the right hand side of this relation immediately leads to the error bound (13). As w 
is not uniquely determined by the equation y — M.(y) = A m u, it is convenient to introduce 
the notation 

U(M, m, y) := inf { \\u\\t*° : y - M(y) = A m u}, (15) 
and for an input class V 

U(M, m, V) := sup U{M, m, y). (16) 

yd 



We are interested in applying (13) to sequences y that arise as samples y n = x (nr) of a 
bandlimited signal x as above. To compare the error bounds for different values of r, one 
could consider the class = {y = (y n )n& '■ y n = x(nr) for some r} and work with the 
constant £/(M, m, V^). However, it is difficult to estimate £7(M, m, ^ x ') in a way that 
accurately reflects the detailed nature of the signal x. Instead, we note that for < fi, 

one has W x > C where = {y = (y n ) ne z : ||y||^*> < /J>} as defined above. This leads to 
the bound 

\\e\\ Loo <U(M,m,y,)\\^\\ L iT m . (17) 



In (17), the reconstruction kernel (p is restricted by the r-dependent admissibility condi- 
tion (|2). However, if a reconstruction kernel (po is admissible in the sense of ^ for r = r = 
1 / po, then it is admissible for all r < r . This allows one to fix p — (l + e )Pcrit f° r some small 
e > 0, and set <p = <£> . Now Bernstein's inequality^ implies that Hv^o^lU 1 — (^Po^H^olU 1 

1 If /is supported in [—A, A], then ||/'||lp < 2TrA\\f\\ L p for 1 < p < oo. 
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since <p is supported in [— y, y]. This estimate allows us to express the error bound natu- 
rally as a function of the oversampling ratio A defined in (pi), as follows: 

Let e and ipo be fixed as above, let M be a given quantizer function, and let m be a positive 
integer. Then for any given f2-bandlimited function x with < /x, the quantization error 

e = e\, as expressed in M) in terms of r = t— ^ — , can be bounded in terms of A: 

||e A || L ~ < U{M 1 m^,)\\^\\ L ^ m {\ + t) m \~ m . (18) 

As the constants are independent of A, this yields an 0(X~ m ) bound as a function of A. 
Note that any dependency of the error on the original bandwidth parameter Q has now been 
effectively absorbed into the fixed constant H^oIIl 1 - in f&ct, this quantity need not even 
depend on Q; a reconstruction kernel can be designed once and for all corresponding to 
Q = 1, and then employed to define <po(t) := ftif^ftt), which is admissible and has the same 
L 1 -norm as <Pq. 

2.3 Exponentially accurate EA modulation 

The 0(A _m ) error decay rate derived in the previous section is based on using a fixed 
EA modulator. It is possible, however, to improve the bounds by choosing the modulator 
adaptively as a function of A. In order to obtain an error decay rate better than polynomial, 
one needs an infinite family ((M mi m))™ of stable EA modulation schemes from which 



the optimal scheme M mopi (i.e., one that yields the smallest bound in (18)) is selected as a 
function of A. This point of view was first pursued systematically in [3]. Finding a stable EA 
modulator is in general a non-trivial matter as m increases, and especially so if the alphabet 
A is a small set. The extreme case is one-bit EA modulation, i.e., when card(.A) = 2. The 
first infinite family of arbitrary-order, stable one-bit EA modulators was also constructed in 
[5j. In the one-bit case, one may set A = {—1, +1} to normalize the amplitude, and choose 
\l < 1 when defining the input class ^ = The optimal order m opt and the size of the 
resulting bound on Ije^ depend on the constants U(M m , m, < y A( ). 

There are a priori lower bounds on {7(M, m, V M ) for any < fi < 1 and any quantizer 
M. Indeed, as shown in [7j, one obtains a super-exponential lower bound on C/(M, m,^) 
by considering the average metric entropy of the space of bandlimited functions, as follows. 
Define the quantity 

U m Q) :=inf E/(M,m,V) (19) 

and let 

:= -fx : supp iG [—1/2, 1/2], 1 1 2,<x> < //}. (20) 
Then, as shown in [Ul E] , one has for any one-bit quantizer 

sup{||e A || L ^ : x G X M } > M 2-\ (21) 



which yields, when used with (18), the result that U m (^^) > M (mc) m for some absolute 
constant c > |7j. Here by A > s B, we mean that A > CB for some constant C that may 
depend on s, but not on any other input variables of A and B. 
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For the family of £A modulators constructed in [3], which we will denote by (D 



vOO 

m) m= n 



the 

\ / nt _L ' 

best upper bounds on L^D™, m, y M ) are of order exp(cm 2 ), resulting in the order-optimized 
error bound ||eA||z,°° < exp(— c(log A) 2 ), which is substantially larger than the exponentially 



small lower bound of (21). The first construction that led to exponentially accurate EA 



oo 

m=l' 



modulation was given later in [7]. The associated modulators, here denoted by (Sm) 
satisfy the bound 

U{S m ,m,\)< (m«) m , (22) 



for a = a(/i) > 0. Substituting (22) into (18) and using the elementary inequality 



min m m a m < e 

m 



-ct/e 



with a = A/(a7r(l + e)), one obtains the order-optimized error bound 

sup{||e A ||i°° : x E X^} < 2 _rA , 



(23) 



(24) 



where r = r(/x) = (a7re(l + e) log 2) _1 . On the other hand, the smallest achievable value of a 
is shown to be 6/e, which corresponds to the largest achievable value of r = r max ((9m)m=i) ~ 
(67rlog2) _1 0.076. Observe from (21) that it is impossible to achieve an exponent r > 1. 



H, 



In [7j, the 9m were given in the form (11) where the causal sequences H,g e £ l with 
= 9o 



1 depend on m, and are related via 

H = A m g. (25) 
Define h := 8^ — H, where 5^ denotes the Kronecker delta sequence supported at 0. Then 



(11) can be implemented recursively as 



v n = (h * v) n + y Tl 



Qn- 



(26) 



If q is chosen so that the resulting v is bounded, i.e., if this new scheme is stable, then 
u := g * v is a bounded solution of (pi), and 



\u\ 



< IMHM 



(27) 



(28) 



The significance of the more general form (11) giving rise to (26) is the following: If 

ll% + IMU<2, 

then the greedy quantization rule 

q n := sign((/i*v) n +y n ) . (29) 
leads to a solution v with ||f||oo ^ 1 ; provided the initial conditions satisfy this bound. 



However, for the canonical form rt8j), the condition (28) clearly fails for all m > 1 



For \\y\\oo < < 1, a filter h will satisfy (28) and hence lead to a stable scheme when 



ll^lli < 2 — /i. Note that (25) together with the fact that gel" 1 implies that Yl hi = 1 and 
hence II h\\ i > 1. 
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In view of the preceding considerations, we are led, for each m to the following minimiza- 
tion problem: 

Minimize ||#||i subject to 5 (0) -h = A m g, \\h\\i < 2 - p.. (30) 



It was shown in [7J that if a sequence of filters satisfy the feasibility conditions in (30) 
for the corresponding m G N, the diameter of the support set of must grow at least 
quadratically in m. 



The minimization problem (30) was not solved in [7J; rather the author introduced a class 
of feasible filters h = hS m > which were effective in the sense that they lead to an exponential 
error bound with rate constant r as above. These filters are sparse, i.e., they contain 
only a few non-zero entries. Indeed, each hS m > has exactly m non-zero entries, which can be 
shown to be the minimal support size for which can satisfy the feasibility conditions. We 
shall call such filters minimally supported. Note that if h has finite support and \\g\\i < oo, 
then g has finite support. We make the following formal definition: 

Definition 2.1. We say that a filter h = 5^ — A m g, for a finitely supported g, has minimal 
support if | supp h\ = m. 

The goal of this paper is to find optimal filters within the class of filters with minimal 
support. 

2.4 Filters with minimal support 

As the filter 6'°' — h arises as the m-th order finite difference of the vector g, its entries have 
to satisfy m moment conditions. This implies that the support size of h is at least m, as 
advertised above. 

For filters h with minimal support 

m 

h = J2d^\ (31) 
i=i 

the moment conditions lead to explicit formulae for the entries dj in terms of the support 
of h, where 1 < n\ < n2 < ■ ■ ■ < n m [7J. Here the condition that n\ > 1 follows 
from the strict causality of h. 
Indeed, one finds 

m 

n 



dj = II (32) 

J J- J- n ■ — n ■ 



n,- — n, 



Here the notation J]', and analogously Y,' \ indicates that the singular terms are excluded 
from the product, or the sum respectively. By definition, if m — 1, one has d\ = 1. 
The condition ||/t||i < 2 — // then takes the form 



m a" 

EIT 



. \rii - rij 
=i i=i 1 j 1 



<2-/i. (33) 
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Furthermore, explicit computations lead to the identity 



I0II1 



n 



ml 



(34) 



In this notation, minimization problem (30) takes the form 



Minimize 



ll,=i rij 



ml 



over {n = (m, . . . , n m ) E N m : ([33} holds and 1 < m < ■ • • < n m } (35) 



For 11 = 1, Problem (35) has a solution only for m = 1, and we find h = but for 
fi < 1, the problem has a nontrivial solution for all m. That is, we can find rij, j = 1, . . . , m, 
that satisfy (33). In particular, for rij (a) = 1 + a(j — 1), one easily sees that 



m m 



hmyn 

J=l i=l 



m la) 



rii{a) - rij{cr) 



1. 



(36) 



So for every 11 < 1, n(cr) satisfies constraint (33) for all a large enough. 



Furthermore, any minimizer n of problem (35) must satisfy ri\ = 1. Indeed, otherwise 



rij > 1 for all j and we can define n by rij = rij — 1 > 1 for all j — 1, . . . , m. Calculate 

m rn 



En 

i=l i=l 



nj — rij I 



m m 1 n 1 m m ' 



j=l i= 



^ \m - rij\ 



<2-n 



and 



117=1 h J nr=i w j 



ml 



ml 



So n cannot be a minimizer. 

Hence, we can fix ri\ = 1, which reduces problem (35) to minimizing 

m 

i=2 

over the set {n = (ri2, . . . , n m ) G N m_1 |l < n2 < ■ ■ ■ < n m } under the constraint 



(37) 



(38) 



(39) 



7ft 

En' 



n,- — 



<7, 



(40) 



where again ni = 1. The factor m! in the denominator has been absorbed into the definition 
of 1] to simplify the notation. Furthermore, we have set 7 = 2 — ri, as the considerations that 
follow make sense for arbitrary 7 > 1 and not only for 7 < 2. 

Notational Remark: All quantities in the derivations below depend on m. We will sup- 
press this dependence unless it is relevant in a particular argument. 
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3 The relaxed minimization problem for optimal filters 



The variables rij correspond to the positions of the nonzero entries in the vector h, so they 
are constrained to positive integer values. We will first consider the relaxed minimization 
problem without this constraint; this will eventually enable us to draw conclusions about 
the original problem. Thus the variables n 3 - G N will be replaced by relaxed variables 
xj G K + . Furthermore, it turns out to be convenient to replace the index set {1, . . . , m} by 
{0,...,m-l}. 

The relaxed minimization problem is specified as follows: Minimize 

m— 1 

r/(x) := J] Xj (41) 

i=i 

over the set D = {xG IR m_1 |l < x\ < X2 < ■ ■ ■ < under the constraint 



m _l m-l 

/W-EIl't^| £ 7, (42) 

■i—n i—n 1 ' J 1 



j=0 i=0 

where x = 1. 

Observe that / is defined and smooth in the open domain D. The following monotonicity 
property for / is important in making inferences from the relaxed to the discrete minimization 
problem. Let r(x) be given by rj(x) = j = 1, ... ,m — 1, and set F(r) = /(x) for x 
such that r = r(x). 

Lemma 3.1. The function F(r) is strictly decreasing in each variable Tj. 
Proof. A simple calculation shows that 

m _im— 1 m-l 1 1 

nr) ^11 , + ,, +2 ... rj _ 1 n r^^. («) 

j=Q j=0 1 j=0 i<j ^ ^ J i>j r j+1 r j+ 2---ri 



from which the monotonicity is immediate. □ 
Definition 3.2. /fx.yeD and 1 < — < • • • < ym ~ 1 , we say that y is subordinate to x. 

Clearly, y is subordinate to x if and only if r 3 -(x) < Tj(y) for j = l,...,m — 1, so 
Lemma 3.1| is equivalent to the following: 

Corollary 3.3. If y is subordinate to x and x/y, then /(y) < /(x). 



If x is a minimizer of the constraint optimization problem (|4JJ), (|42j), then /(x) = 7. 
Indeed, for a proof by contradiction, assume that x is a minimizer and /(x) < 7. Then for 
t G [0, 1), we can define Xj(t) = (1 — t)xj + txQ. Since / ox is continuous in t and 

/(x(0)) = /(x) < 7, (44) 
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there exists t > such that / (x(£)) < 7. However, the function 



m— 1 



is decreasing in t, so 



3=0 



and x cannot be a minimizer. Hence we can replace constraint (|42|) by the equality 

m _l m-l 

/w = EIT 



m _l m— 1 



j=0 i=0 



I 2/ 2 *^ J I 



7- 



As we now show, this equation defines a smooth manifold within D. It is enough to 
that V/ ^ 0. Note first that 



d x k 
3x k \x k Xj J 

Now calculate for j 7^ k using this fact 



1 



1 



X\z %j \%k % j 



m-l 

dxk^rK I 



1 Xi 



i=0 



^m— 1 

n 

\ i=0 

(-i) J 



6 i> 



where we set from now on 



m— 1 



/ 1 



2=0 J 



Note that (— iybj(x.) is always positive. 
Furthermore, for j = k, 

m-l 

a tt' Si 

ft i=0 



m—l 

E 

z=o 



77 (x) 



Hence 



5/ 

dxk 



771—1 



E' ,7^7 ((-l)*6*(x) + (-l)^(x)) • 

J=0 J 



For A; = m — 1, all terms in the sum are positive. Hence 
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df 

and so {x|/(x) = 7} is a manifold within D. 



< 



(54) 



We now show that the infimum of 77 subject to (47) is attained in D. Let 7/0 
inf r/(x) and let x^ G D fl {/ = 7} be chosen such that lim T](y^- n '] 



xeD,/(x)= 7 



i] . As before, 



we set x, 



(n) 




1. 



We first show that ~x> n > is bounded. Define M := sup 77 (x^). Then for each n, 



r(n)\ 



x^ x i < r, (xW) < M. 



(n)> 



(55) 



as, for each i, 1 < x| n ^ < x^l x . Since M < 00, it follows that x^ n ^ is bounded. We conclude 
that x( n ) must have a convergent subsequence x^ nfc ^ — > 

Now x^ 00 ) cannot lie on the boundary of D. Indeed, for any < j ^ k < m — 1, we have 



m— 1 
/ 



7 = /(x^) > n 



(n) 



j=0 



I ( n ) ( n ) 

j 3/ 3/ j 



> 



M 



m-2 



(«) 



Mi' 



(56) 



which implies that — 



> A/ m_ 2 > 0. It follows that x*™) stays away from the 
boundary of D, which implies that x^°°^ G D. Thus, problem (41), (47) must have at least 
one minimizer x m j„ = x^°°^ in D. Note that a priori, there can be more than one minimizer. 

As {x|/(x) = 7} is a manifold within D, every minimizer x min = {x\, . . . ,x m _i) of the 
constrained optimization problem given by (41) and (47) solves the associated Lagrange 
multiplier equations, i.e., there exists v = z/(x min ) ( 



uVf]{^ min ) + V/(x 

min) 



such that 

= 0, 

= 7- 



(57) 
(58) 



Combined with (53) and the relation g~^(y) = ^v(y)i the Lagrange multiplier equations 



(57), (58) take the explicit form 



m— 1 



£' — ((- 

J= J 



■D k hUr 



-lybj^min)) = V, 
fip^-min) 7 



(59) 
(60) 



for k — 1, . . . , m — 1 and xq = as before. 

Note that any critical point x cr j 4 of the minimization problem for rj on D solves equations 
(59 ), (60 ) for some v. In the following section, we will show that in fact r\ has a unique critical 
point in D. 
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4 Solution of the relaxed minimization problem 

Theorem 4.1. The minimum value of rj on the manifold {/ = 7} in D is given by 

sinh(2m/3) 
V = Vmin = (2sinh/3) 2 — !cosh/3 

where (3 = /3 (771,7) ^ s the unique positive solution of the equation 

cosh ((2m - 1)(3) 



7- 



cosh (3 

The minimum value r] min is attained at the unique point x 

1 



1 + 



2 sinh 2 p 



mm \X± 5 • 

1 + z j), j = l,...,m-l. 



,x m -i), where 



(61) 



(62) 



(63) 



Here Zj = cos (^-^7r) , j = 1, . . . ,m — 1, are i/ie zeros 0/ t/ie Chebyshev polynomial of the 
second kind of degree m — 1 . 



Proof. The minimization problem (41), (47) assumes its minimum in D, so there must be 



at least one critical point x crit = (xi, . . . ,x m _i) with 1 < x± < ■ ■ ■ < x m -\. 

To prove uniqueness, we will express the associated Lagrange multiplier problem as a 
nonlinear matrix equation and then show using a rank argument, which is established by 



Proposition 4.2, that the equation can have only the solution given by (63). 



As in (59), x crit = (x 1; . . . , x TO _i) must satisfy 



m— 1 



^— {(-l)%(Xcrit) + (-l) J 6 J (x crit )) 

Xk Xj 



V X 



crif ) , 



i=o 



for /c = 1, . . . , m — 1 and, again, bj(-x crit ) 
In matrix notation, the statement reads 



TO— 1 

I 7 1 



IT 

i=0 



(64) 



BU crit )v = i/fx 



1,1,..., if e m™- 1 , v 



where e 

function 5 : M™" 1 -> R(™-i)x"» is given by 



1,-1,1,-1, 



/ 



B(y) 



My) 
2/1-2/0 

My) 

J/2— 2/0 



TO — 1 

^ My) 



3=0 

61 (y) 



My) 

2/i-% 2/1-2/2 

TO — 1 

My) 



£' 



2/2-2/1 2/2-2/,' 
j=0 



My) 

Um-l—yo Vm-l—Vl 2/m-l-2/2 



My) 



My) 



(65) 

' m and the matrix-valued 



2/1-2/m-i 
ftm-i(y) 

2/2-2/m-l 



TO — 1 

fam-i(y) 



£' 

j=0 



2/m-l— 2/j 



(66) 
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where y = . . . , y m -i) and as before y = 1. 

For given y — (yx, . . . , y m -i) let p y (s) be a polynomial such that 

m— 1 



(67) 



For definiteness, we normalize p y (0) = 0. Let T be a positively oriented circle in C of radius 
R large enough to enclose all y^s, including y = 1. We now calculate the integral 



Py(z) 



2?ri / (z - y fc )(z - y Q )p'(z) 



dz, k = 1 m — 1 



(68) 



in two different ways. 

Firstly, letting R — > oo, we see that Ik = —■ Secondly, we compute the integral using the 
residues at yj, < j < m — 1. For the residue ii, at ?/,, j 7^ fc, we obtain 



Vj - Vk 



At we have a double root in the denominator of the integrand in (68), so 

V 



Ri 



Py{ z ) 



m—1 

Yl(z-Vi 

, i=0 



PyiVk 



z=Vk 



m—1 

n (vk - Vi 

i^k 



m—1 

n {vk - Vi 

m-l i=0 

PyKVk) 2^ 

j=0 I m-l 

3+k ( n (vk - yi 

i=0 
,i+k 



m—1 



■i) m - 1 E' 



j=0 



b k (y) 

Vj - Vk 



PyiVk 



Summing the residues, we conclude that for k — 1, . . . , m — 1: 

m—1 



:-i) 



m—1 



rn 



1 



or equivalently 



Vj ~ Vk 



(h(y)Py(yk) + bj(y)py{yj)) 



:-v, 



m 



(69) 



(70) 



(71) 



(72) 



(73) 



where p y = (p y (y ) , p y (yi) , ■ ■ ■ ,P y (y m -i)- 

The normalization p y (0) = plays no role in the above calculation, and so (73) also holds 
for p y + e. Hence, the vector e lies in the kernel of B(y) for any y. In Proposition 4.2 we 



will show that dimKer5(y) = 1, and hence Ker5(y) is spanned by e. In particular, this 
shows that z/(x cr j 4 ) 7^ 0: Otherwise, v would be collinear to e, which is impossible. 
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Specifying y = x crit , one obtains 



B(Xcrii)Px cnt 



m 



and it follows that 



5(x crit ) [m(-l)"V(x cri i)p Xcr J = z/(x crit )e. 



By ( 65 ) , v also solves ( 75 ) , and thus 



for some constant c. 
Set 



v - m(-l)"V(x cri4 )p Xcrit = ce 



g (s) = m(-l)"V(x cri4 )p Xcrit (s) + c. 



Then q is a polynomial of degree m with critical points at the j 



(74) 
(75) 

(76) 

(77) 
1 such 



that q(xj) = (~iy, j = 1, 



, m 



1. As g cannot have any more critical points and must 



be monotonic for x > a? m _i, then ultimately, it will change sign and there is a unique point 
%m > %m-i such that q(x m ) = —q(x m ^.\) = ( — l) m . Hence the polynomial u given by 



U[S 



(-l) m q 



(s + l) 



(7E 



T m (s) 



has the equi-oscillation property, and we conclude by Proposition A.l that u(s) 
That implies, that if z m , j = 1, ... ,m — 1, are the extrema of T m - i.e., the zeros of the 
Chebyshev polynomials of the second kind of degree m — 1 - then the extrema of q are given 
by 



x m 1 



;i + *,■) + !■ 



Thus all critical points x 



crit 



[Xl, 



j %m—l, 



are given by 



:= 1 + ^(1 + ^^- 



(79) 



*0) 



for some constant K. It follows from Lemma 3.1 that f(x.(K)) is strictly monotonic in K 



i.e., different values of K correspond to different values of 7. This proves that 77 has a unique 



critical point on {/ = 7} in D, and, in particular, that x min is unique and given by (63). 

We now compute K = K(m,j). The calculation uses several facts about Chebyshev 
polynomials and their roots, which we have collected in the Appendix. Let (3 = {3(m, 7) > 
be defined through the relation 

K= ?-=— . (81) 

2 sinh 2 V ' 
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Noting that Zj 
/(x) 



-z. 



m—ii 
m—l 



we obtain from Lemma IA.2 



n i + if (i + Zj) , ^ n 1 + ^ + Zi 

l\ K\zi-Zo\ 11 



3=1 i=0 



K\zi — Zi 



im-l m ~ 1 



m 



n 



1 + AT(1 + 



i=i 

m—l 



K 



™ 1 nm-l 

E- 

3=1 



(1 - Zj) fr 1 1 + A(l + 



r?) 



1 Zm—i 



m—l 



n 

i=0 



1 "I - Zm—j 



A 



ym-l m ~ 1 



m 



n 



i=l 



A' 



+ 1-2, 



1 + |^T + A(l-z m _,)_ 



Now 1 + ^ = 1 + 2 sinh 2 (/3) = cosh(2,5) and 



im-l m ~ 1 



m 



n 

i=i 



A 



2 m 



t 1 + 



A 



1 m ; / , sinh(2m/3) 
-T;(cosh(2/3)) = ^-^ 



m sinh(2/3) 



Furthermore, differentiating (162), we obtain for z = cosh(r) 

1 ' T^(z) m coth(mr) — coth(r) 



sinh r 



Hence 



1 ^ 1 + Zj 

,,,„„, > ^ / 1 + cosh(2ffl \ 



m—l 



1 - (m - l)(cosh(2/3) - 1) + (cosh 2 (2/3) - 1) ^ 



1 - (m - l)(cosh(2/3) - 1) + sinh 2 (2/3) 
m (1 - cosh(2/3) + sinh(2/3)coth(2m/3)) 



J= i cosh(2/3) - ^ 

mcoth(2m/3) - coth(2/3) 
sinh 2/3 
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Combining (92) and (86) yields 

sinh(2m/3) - cosh(2/3) sinh(2m/3) + sinh(2/3) cosh(2m/3) 



7 = /(x) 



sinh(2/3) 
sinh(2m/3) - sinh((2m - 2)/3) 

2sinh(/3) cosh(/3) 
2cosh((2m - 1)0) sinh(/3) 

2 sinh(/3) cosh(/3) 
cosh((2m- 
cosh(/3) 



(93) 
(94) 
(95) 
(96) 



which proves (62). As ^fu^ 1 ^^ is strictly monotonic in /3, /3 > is uniquely determined 
from 7. Of course, this fact also follows from the uniqueness of K proved above. 



Now finally, using (86), we write 



m— 1 to— 1 / -. 

n(i+^(i+^))=^ m - i n {k +i+z * 

i=0 i=0 ^ 



sinh(2m/3) 



(2sinh(/3)) 2m - 1 cosh(/3) 



(97) 



□ 



It remains to show that B{^ crit ) has rank m — 1. We will show, more generally, that 
B(y) has rank m — 1 for an arbitrary y = (yi, . . . , j/ m _i), as long as 7^ % for i ^ j in 
{0, 1, . . . , m — 1}. As before we set y = 1. The proof of Proposition 4.2 below goes through 
without this restriction on yo, but this more general fact is of no consequence for the results 
in this paper. 

Factor out bj(y) from the j-th column, j — 0, . . . m — 1 and extend the resulting matrix 
to an m x m square matrix B(y) by adding a row that is the negative of the sum of all the 
other rows, as follows. 



/ TO— 1 

E' — 



B(y) 



1=0 



yo-yi 



2/0-2/1 



1 



yi-yo 
1 

2/2-2/0 



TO— 1 

E' — 



1=0 



Di-Vl 



1 



2/2-2/1 



1 



1 



2/0-2/2 
1 

2/1-2/2 

TO— 1 

E' — 

Uo " 



1 



1 


yo 


-ym-i 




1 


yi 


-2/m-l 




1 



2/2 -2/m-l 



V 



2/m-l— 2/0 



2/m-l— 2/1 2/m-l-2/2 



TO— 1 

E' 

Z=0 



1 



2/m-l— 2/! 



(98) 



/ 



Clearly, rankS(y) = ranki?(y). We prove that -B(y) has rank m — 1 by explicitly showing 
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that B(y) is similar to the Jordan block 



J 



( 1 

1 



V ; ; ; 



Proposition 4.2. Form > 2, B(y) has the Jordan decomposition 

B(y) = P(y)JP(y)-\ 



where 



P(y) 



( My) 6o(y)(yo-2/m-i) 

My) 6i(y)(yi-ym-0 

62 (y) &2(y)(2fe-ym-i) 

(y) & m -2(y)(z/ 

V 6m-i(y) 



My) 
My) 
My) 



(.yo 


-2/m 






(m— 


1)! , 


(2/1 


~2/m 






(m- 


1)! , 


(2/2 


-y m 





(m-l)! 



6 T n-2(y) (ym - 2 " ym - ir " 1 







(m-l)! 



/ 



(99) 



(100) 



(101) 



Here the bj(y) 's are defined as in (51). 

Proof. The matrix P(y) is of the form DiVD 2 , where D\, D 2 are invertible diagonal matrices 
and V is a Vandermonde matrix. Hence, P{y) is invertible and the proof of ( 100 ) is equivalent 
to showing that f?(y)P(y) = P(y)J, that is, for < j, n < m — 1, 



v^ 1 My) (y k -y m -i) n , ^ My) (^ -y m -i) ra _ , , ^ -y m -i) n 1 nn9 , 

where pjyy = 0. 

The proof is based on the counterclockwise integral defined for all t e C 

1 r ( z — t) n 

J m ,n{t) = ^-f —dz, < n < m - 1 (103) 

r II (* - y<) 

i=0 

over a circle Y of radius i? large enough that it encloses all y/s. Letting R — » oo, we see 
that J myn = <^_/ m _ 1 - ) independent of t. On the other hand, note that the residue at y^ is 
{-l) m \{y){yk-t) n . Hence 

m— 1 

^ (W -X) = = (-1)™- 1 £ ^Hf* - ( 104 ) 

fc=0 
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Now 



My) 

Vk-Vj 
m—l 



E 

1=0 



My) 



for j 7^ k 
for j = k 



Hence, differentiating (104) with respect to yj, leads to the identity 



* * ? / 7_ — 1 1 • ' * ? / 1 — 7 / ■ 



&i(y) 



fc=0 

Mi 



i=0 



W - Vi 



(105) 



(106) 



Letting t — > y m _i, one obtains (102). 



□ 



5 Asymptotics for the relaxed and the discrete mini- 
mization problem 

In the following proposition, we evaluate the dependence on m of the solution x = x^" 1 - 1 of 
the relaxed minimization problem. For any fixed j, we show that Xj converges as m — > oo, 
and we compute the limit. 



Proposition 5.1. (a) For K = K(m, 7) as in (80) 



2(m-l) 2 2m 2 , „ N 

- 1 < K < - — j - . (107) 



(cosh 1 7) 2 (cosh 1 7) 2 

2 

(b) Set cr := -; — — — -2 . Then for all m and all 1 < j < m — 1, 

( cosh 1 7 J 

xj m) < 1 + cTj 2 . (108) 

(c) For any fixed j > 1, 

lim z$ m) = 1 + aj 2 . (109) 



(d) 

(r/(x(™))) 1/m 1 
lim ^ ^— = -1 , 2 = -3. HO 

m^oo m z (cOSh 7) 2 7T Z 



Proof. We first provide bounds on /3 defined in (62). For a lower bound, write 



cosh(2m — 1)8) 

7 = —^- L = cosh(2m/3) - sinh(2m/3) tanh 8 < cosh(2m8). (Ill) 

coshp 
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For an upper bound, we have 
cosh(2m - l)/3) 



cosh (3 
We obtain the bounds 



cosh((2m-2) / 9) + sinh((2m-2)/3)tanh/3 > cosh((2m-2)/3). (112) 



— cosh 1 7 < (3 < cosh 1 7. 

2m 2m — 2 



This implies the upper bound for K 



K 



1 



< < 



2m 2 



2sinh 2 /3 - 2/3 2 ~ (cosh'S) 2 ' 
For the lower bound on K, we have by an elementary estimate 



K 



I > J_ _ 1 > 2(m-l) 2 _ L 



2sinh 2 /3 '- 2/3 2 ~ ^ (cosh'S) 2 



This proves (a). 
Also 



,J m ) 



which proves (b). 



From (107) 



, 77T \ 4m 2 / ?7T 

1 + 2/Ysin 2 — < 1 + = — 

2m J ~ (cosh-^) 2 V 2 ™ 



lim K{m r ] 



1 + af 



m->oo m 



(cosh 1 7) 2 ' 



and so 



lim x^ m ^ = 1 ~\~ 

m— >-oo 3 (cOsh _ 7) 2 m— s>oo 



r ,, 2 • 2 / 
lim 2m sin | 

2m 



1 + of 



which proves (c). 



Finally, from (113) we see that 

lim 2m/3(m, 7) = cosh -1 7, 



and hence from (61) 

m— >oo m 



lim 

m— >oo 



sinh(2m/3) 



l/m 



m 2 V.(2sinh/3) 2m - 1 cosh/3 / 

1 \ /2sin/3sinh(2m/3) x 1/m 



lim . ,-, 

■m^oo ^ 47772 ^ 

lim ^ (m ' 7) 



cosh /3 



2m 2 (cosh 1 7) 2 ' 
which proves (d). This completes the proof of the Proposition. 



(113) 



(114) 



;ii5) 



(116) 



(117) 



:ii8i 



;ii9) 



(120) 

(121) 
(122) 

□ 
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Motivated by the above proposition, let us define 



Wj := 1 + aj" 



j > 1 • • ■ 



(123) 



and an associated vector sequence w 



(m) 



(wx, • • • , u> m -i), m > 2. We know that for each 
j > 1, scV"'' — >■ as m — > oo. The following lemma provides a nonasymptotic relation 



between x^ m ^ and which will be utilized in proving Proposition 5.3 and Theorem 5.5 
Lemma 5.2. w^ m ^ is subordinate to yS m \ 

we need to show that for < j < m — 2 and w^ 1 



Proof. By Definition 



3.2 



(m) 



(m) 
X j+1 



> 



(m) 



(m) 

x) 



(124) 



that is, 

(l + a(j + l) 2 ) 
or equivalently 



1 + 2/rsin 2 



2m 



> (1 + ctj 2 ) 1 + 2Ksm 



(j + 1> 
2m 



(125) 



+ 



o-(2j + 1) - 2K (sin 
2aiT ( (j + l) 2 sin 2 



(j + 1)tt 



2m 



sin 



2m 



■2 • 2 

j sin 



J 71 " 
2m 

(j + J> 
2m 



> 0. 



(126) 



We show that both the se summands are nonnegative. 

By Proposition 5.1 a), K < 2 ™ 2 U , and so for the first summand, it is sufficient to show 
that 

4m 2 ( . o ( ( j + l)7r s 



(2j + 1) 



7r z 



sm 



2m 



sm 



J 71 " 
2m 



> 0. 



(127) 



Indeed, by standard trigonometric identities 



4m z 



7T Z 



sm 



(j + i: 

2m 



71 



sm 



J7T 

2m 



4m 



7T 



sm 



(2j + i)^ s . n m 



2m 



\2mJ 



<2j + 1, (128) 



which proves (127). 

On the other hand, the positivity of the second summand follo ws f rom the fact that the 
function sin ^ is decreasing on [0, |]. This completes the proof of (126) and hence the proof 
of the Lemma. □ 

The above results for the relaxed minimization problem allow us to draw conclusions for 
our original problem with the constraint that the filter locations are all integers. 



With n\ 



(m) 



(m) 



1, we seek an inte ger s equence n 



(m) 



in 



(m) 
2 J 



(m)> 



j rim 



such that 



n (m) ig 

subordinate in the sense of Definition to ^ : = (1 + K(l + Zj ))™~\ the solution 
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of the relaxed minimization problem (41), (42). By Corollary 
so n( m ) satisfies (40). Note that for the n 



(m~ 



3.3 



f(n^) < /(xM) < 7 , 
1 m of 



Section 



2.4 



while for the x\ s we retain the labels 7 = 

3 J 



s we use the original index set j 

, m — 1. In this section, we 



work with the specific integer sequence n 



(m) 



n 



(m) 



(m) X j 



(m) 



(m) 

i-i 



(n^™"*, . . . , nffl) defined recursively by 
j = l,...,m-l, (129) 



where n\ 



(m) 



,( m ) 



1 as above and \s\ denotes the smallest integer greater or equal to s. 



This sequence is minimal amongst all integer sequences subordinate to x*™) in the sense that 
if k = (&2, • • • , k m ) is any integer sequence such that 1 < -j^y < -• - < — - i 1 " " " ! "' ! 



-j^y, then kj > n) 



for all j = 2,...,m. Indeed one has k 2 > x*f a \ which implies k 2 > [^i mJ ] 
assuming by induction fcj > , one obtains 



n 



(m) 



and 



X 



(m) kj 



(m) 



X 



3-1 



> 



X 



(to) H j 



(m) 



(m) 



X 



3-1 



(m) 



(130) 



r (m) 

of n\ 



We will next derive an asymptotic formula, analogous to Proposition 5.1[ c), for the limit 
asm-) oo. 



Proposition 5.3. Let a > f. Then, for all j > 1, 



lim n 



(m) 



i+MO'-i) 2 . 



(131) 



Proof. We will prove the statement by induction on j. The case j = 1 holds by definition. 
Assume that (131) holds for some j > 1. 



We first consider the case when a is an integer. Since is integer valued and \a~\ = a 



(131) is equivalent to saying that n ™ = 1 + o~(j — l) 2 for all sufficiently large m. Now by 
Lemma 5.2| (see (124)) we have 



1 + a f > (i + G {j _ i 



(m) 



n 



(to) 



;i32) 



which, together with ( 129[ ), implies 1 + aj 2 > nl^[. Since > 



,(to) 



see Proposition 5.1 'c)), we obtain — >■ 1 + crj 2 , which completes the induction step. 



X 3 



(to) 



and x^ — > 1 + of 



Now assume a > | is noninteger. By Proposition 
we have 



5.1 



c) and the induction hypothesis, 



lim n 



(to) X j 



(to) 



(to) 

i-i 



1 + of 



(133) 
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Since the function [•] is continuous at every noninteger, our problem reduces to showing 
that 

1 + °3 2 



\a]f<(l + \a](j-l) 



l + f7 (i-l) 2 



< i + wv 



for all j > 1 and a > |, for then we have 



lim n 



(m) x j 



(m) 



(m) 



X 



3-1 



lim 

m— »oo 



(m) x j 



(m) 



(m) 



lim n 



(m) 



(134) 



(135) 



which would complete the induction step. 



To show (134), first note that 



1 + Crf 



l + cr(j-iy 



WW 



(H-q)(2j-l) 

1 + Or(j - 1)2 



(136) 



which immediately yields the second inequality for all j > 1 and noninteger a > 0. The first 
inequality in (134) is also immediate for j = 1, as the right hand side of (136) reduces to 
1 — ([cr] — er), which is strictly positive. Now note that 

1 + a(t - l) 2 > 2at + 1 - 3(T, for all t G E, (137) 

since the right hand side is the equation of the line tangent to the parabola 1 + a(t — l) 2 at 
t = 2. Now, for cr > 2, we have 



(W - v)(2j - 1) 2j-l 
l + a(j-l)2 - l + 2(j- 

5 



< 



2j - 1 
4j-5 



< 1, for all j > 2. 



(138) 



On the other hand, for | < a < 2, we have 



(M - ^)(2j - 1) < , 



3 (2j 



l + a(j-l) 



1 + I(J-1) 2 



< 



3„ _ 3 
4 



1^ 



± < 1, for all j > 2. 



Hence the first inequality in ( 134 ) holds for all j > 1 and noninteger a > 



(139) 



□ 



Remark: For j = 1 and j = 2, we have n^' 



(m) 



1 and n^" 1 ' 1 



[x^ m ' > ] so that the formula 



(131) is actually valid for all a > because of Proposition 5.1[ b) and (c). On the other 
hand, for j 



n 



(m) 



3 and 1 < a < |, we have n^" 1 ' 1 — )■ 8 instead of 9. To see this, note that 



— > 1 + [c"| = 3, so that 



lim n 



(m) 2-2 



(m) 



X 



(m) 



3(l + 4cr) 

1 + CT 



G (7.5,8). 



(140) 



Similarly, for 1 < a < |, n^" 1 ^ — >■ 17 instead of 19. As shown in Figure [l| the pattern 
becomes more complicated for higher values of j and the interval has to be subdivided. A 
similar pattern is present also for < o < 1. We shall not explore this here; we again refer 
to Figure [Tj 
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60 - 
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i = 4 

:'t — rOO 4 * 
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Figure 1: Numerical comparison of lim x^™\ with lim nf 1 ^ as a function of a for j = 
2, 3, 4, 9. For j = 2, these limits equal 1 + a and 1 + \o\ , respectively. For j > 3, notice the 



difference between the cases a > 5/4 and er < 5/4. In the first case, one has lim n) 
1 + [cr] (j — l) 2 , whereas in the latter case this formula is no longer valid. 
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(m) 



Definition 5.4. A sequence of integer vectors \S m ^ = (k?^, . . . , kffl) of increasing length 
m — 1, m = 2,3..., with /(ifeH) < 7 is said to be asymptotically optimal if 



lim 

m— ¥oo 



'rj (k( m )) 
rj (xW) 



l/m 



(141) 



where is the solution of (41), (4-2) as above. 

The relevance of this definition lies in the fact that the quantity lim (//(k^)) 1 /™ 

m— >oo m 

controls the rate of exponential decay for the error bound associated with the sequence of 
filters defined by (k( m )). The precise relation will be seen in Theorem 



5.6 



below. We will 

use the following lemma to assess the asymptotic optimality of our minimal subordinate 
construction n 1 -"^ defined in (129). 



Theorem 5.5. If a , defi ned in Proposition 5.1, is an integer, then the sequence n^ m \ m = 
2, 3, ... , defined by (129), is both asymptotically optimal and subordinate to x^ m \ If a is not 
an integer, no sequence of integer vectors can have both of these properties. 

'' ' — 1 + cr is not an integer, and so for any sequence 
2, 3, ... , of integer vectors subordinate to x^ m \ 



Proof. If a is not an integer, then lim x\ 



k M = (fc M j 



; km rn 



lim sup 



4 m) , r^ri 

> hm sup , ; > lim 



m->oo x 



(m) 
1 



X 



(m) 



1 



> 1. 



a 



Hence k^ m ^ cannot be asymptotically optimal. 

Now consider the case that a is an integer. Then by Lemma 
is an integer sequence subordinate to x^ m \ As in Proposition 
as n*™) is the minimal integer sequence subordinate to x*" 1 ). 



5.3 



5.2 



w 



(m) 



1 ( m ) ^ 

one has Xj < < Wj, 



(142) 



(m) 



Next, from Proposition 5.1 a) we see that 

2K 4s 



> 



77T 



7T Z 



Ci 
m 



(143) 



for some constant C\ < oo. Together with the elementary fact that > 1 — C2X 2 for a 

sufficiently large constant C2, this implies that for 1 < j ' < m 2 / 3 and some constant C3 < 00 



Xj = 1 + 2A sin 



J 71 " 
2m 



> l+o- 1 



Ci 
m 



4 m 2 



> (1 + ^J 2 ) 1 



Then for 1 < j < m 2 / 3 and some C4 < 00 



(m) 



< 



< 



1 - 
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c 3 

m 2 / 3 



< 1 + 



2/3 ' 



^3 
m 2/3 

(144) 



(145) 



Now 



n j+1 



(m) 



n 



(m) X j 



(m) ' 



X 



(m) 



< 



n 



(m) 
(m)~J 



+ 1 



77 



x 



(m) 

J 

(m) 



+ 



(146) 



sin x ~> 
x — 



for < a; < | , we 



Combining (143) together with the elementary lower bound 
obtain Xj > C$j 2 , j > 1, for some constant C5 > 0. By repeated application of (146), one 
then obtains for m 2 / 3 < j < m — 1 and some constant Cq < 00 



n 



x 



(m) 
3+1 
(m) 



< 



n 



(m) 



1 



a; 



(m) 



r? (m) 4- 1 

< ... < V 2/3 J 1 



1 



Thus there exists some constant C 7 < 00, such that for all 1 < j < m — 1 



(m) 



< 1 + ^ 4 1 ^6 

^ C5Z 2 ~~ m 2 / 3 m 2 / 3 ' 

i=|m 2 / 3 J+l 



(147) 



< 1 + 



2/3 ' 



We conclude that 



which implies that 



1 < 



77(11 



{m)\ 



m— 1 (m) 



r/(x(™)) 



n 



a;,-( m ) 



m 



< 1 



C 7 



2/3 



lim 

m— »oo 



77(ll 



7/(x('' 



1/m 



1- 



and hence 

n (m) ig 

asymptotically optimal. 



(148) 
(149) 

(150) 

□ 



Remark: For noninteger values of cr, we do not know if there are asymptotically optimal 
integer vectors k^ m ^ (which are necessarily not subordinate to x^ m )) that are admissible, i.e., 
that satisfy f(k^) < 7 = cosh(vr/ v /o : ). At the same time, it is natural to ask how close our 
construction 

n (m) ig 

to being asymptotically optimal, i.e., the value of the limit, as m — > 00, 
of (7/(n( m ))/7/(x( m ))) 1 / m . We shall not carry out an analysis here that is similar to Proposi- 
tion 5.3 to find this limit, but instead provide our numerical findings in Figure [2j 



Optimal exponential error decay for minimally supported filters 

We are now ready to prove the promised improved exponential error decay estimate for the 
XA modulators defined by n^ m \ 

2 

Theorem 5.6. For all 1 < 7 < 2 such that o = - — — — -j is an integer, all one-bit £A 

(cosh 1 7 J 

modulators corresponding to filters minimally supported at positions 1, . . . , nffl are 
stable for all input sequences y with \\y\\oo < = 2 — 7. Furthermore, the family consisting of 
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lim ^ 7 ( ?) (x('"))) 1/ " 

m—tac Tfl 

lim i^nW)) 1 '" 








0.5 



Figure 2: Numerical comparison of lim -^(7/(n (m ))) 1//m with lim (^(x^)) 1 ^™ = a/ 
as a function of a. Top: plotted individually, bottom: their ratio. 



Ti- 



the one-bit SA modulators corresponding to the filters {h^}°° =2 for all orders m gives rise 
to exponential error decay: For any rate constant r < ro := e2 J r ln2 ; there exists a constant 
C = C{r) such that 

HsaIIoo < C2-^ A . (151) 
Proof. Stability follows from the fact that 

n (m) ig 

subordinate to yS m \ which satisfies the 

stability condition /(x^ m ^) < 7. 

Choo se the reconstruction kernel <£> such that the corresponding e as introduced in Sec- 
2T2] satisfies 1 + e < J^. Now let g^ be such that A m g^ = #°) - h^ m \ as in 



tion 



Section 2.3. Then from (18) and (27), we have the error bound 

||_(m)|| ||„,|| II, _ || -r.mi-\ 1 Jm\- 



l e A| 



(152) 



where t> solves (26). 



Recall that our construction yields ||f ||oo < 1- Furthermore, by Theorem 5.5 and Propo- 
sition 



5 . 1 1 (d), we have that 



lim 

m— >oo 



(,,(nW)) 1/ra 



1 



a 

1 V|2 ~~ ^2 



(cosh 7) 2 7T 2 



(153) 
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and hence by (34) 



\g {m) h 



77(11 



{m)\ 



( eay 



m 



;i + (i)) m . 



(154) 



ml \7r 2 / 

Now consider m > M(r) large enough to ensure that the (1 + o(l))-factor is less than 
. Then (152) implies 



,, /ea\ m 



1^0 ||l 



V 7T / 



m 



r 



(155) 



As explained in Section 2J3 we choose, for each A, the filter h^ 71 ' that leads to the minimal 
error bound. Then by a slight variation of (23), we obtain 

\- m <r exp 



< ||y?o||i min 



eer\ ' 



- m r ' 



r 



it r 
e% r 



A 



i>M(r) V 7T / 

which proves the theorem. 

Remark: The smallest integer a such that the stability constraint 



-rX 



(156) 



□ 



< 7 is satisfied 



for some 7 < 2 is a = 6. In this case, Theorem 5.6 yields exponential error decay for any rate 
constant r < ro ~ 0.102. This is the fastest error decay currently known to be achievable for 
one-bit £A modulation. The previously best known bound for the achievable rate constant 
was r ^ 0.088 [H]. 



6 Multi-level quantization alphabets and the case of 
small a 

In this paper we have primarily considered one-bit SA modulators, though our theory and 
analysis is equally applicable to quantization alphabets that consist of more than two levels. 
Let us consider a general alphabet A = Al with L levels such that consecutive levels are 
separated by 2 units as before. For instance, A4 = {—3, —1,1,3} and A5 = {—4, —2, 0, 2, 4}. 
The corresponding generalized stability condition for the greedy quantization rule is then 

IHIi + ||y||oo<£, (157) 

which still guarantees the bound ||f ||oo < 1 (see, e.g., [131 P- 104]). In terms of our filter 
design and optimization problem, the parameter 7 that bounds \\h\\i can be set as large as 
L. The potential benefit of increased number of levels is that large values of 7 correspond to 
small values of a, which in turn yields faster exponential error decay rates in the oversampling 



ratio A as given in (156). There is, however, also an increase in the number of bits spent per 
sample which needs to be accounted for if a rate-distortion type performance analysis is to 
be carried out. 



As seen above (Theorem 5.5 and Figure [2J), our analytical results are available for integer 



values of a only. We will evaluate the performance of multi-level quantization alphabets 
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L 


2 


3 


4 


5 


12 


(average) number of bits/ sample B = log 2 L 


1 


1 ror 

1.585 


2 


2.622 


3.585 


minimum integer value of a 


6 


4 


3 


2 


1 


maximum input signal \\y\\oo 


0.058 


0.490 


0.851 


0.335 


0.408 


achievable error decay rate r = 7r(e 2 0Tn2) _1 


0.102 


0.153 


0.204 


0.306 


0.613 


coding efficiency r /B 


0.102 


0.097 


0.102 


0.132 


0.171 



Table 1: Comparison of exponential error decay rates and their coding efficiency for multi- 
level quantization alphabets. 

based on these values first. For each positive integer a, we can find the minimum integer 
value of L > 2 such that cosh(7r / < L. Alternatively, if L > 2 is specified first, we find 
the minimum integer value of a that satisfies this condition. Given such a (a, L) pair, the 
corresponding bound ||y||oo on the input signal is L — cosh(7r/ v /a). We also compute the 
achievable exponential error decay rate r given by e2 J r ln2 , the average number of quantizer 
bits per sample B := log 2 L, and a coding efficiency figure given by r /B. The results are 
tabulated in Table [TJ It turns out that the smallest value of L which results in a = 1 is 
L = 12. This case also yields the highest coding efficiency figure given by 0.171. It also 
yields a significantly more favorable range for the input signal compared to the one-bit case 
(L = 2). As before, it is easy to check via Kolmogorov entropy bounds that the coding 
efficiency is always bounded by 1. The cases L = 4 and L = 5 are also noteworthy for they 
provide better overall performance, yet still with a small quantization alphabet. 

It is natural to ask what can be said for the case of noninteger values of a, especially as 
a — > 0. For our minimal subordinate constructions, we employ the numerically computed 
values of limm^oo given in Figure 2 as a function of the continuous parameter 



a. As in Theorem 5^, the achievable error decay rate r is given by 

r = — 7tz — lim (158) 

7re 2 ln2 m^oo ^^ n(m) ^i/m v ' 

On the other hand the smallest number of levels L of the quantizer which guarantees stability 
for the greedy rule is given by 

L := [cosh(7r/v/o)l. (159) 
Finally, the coding efficiency as a function of o is 

= — tt, — f — , ] — ^ — ■ lim —j—. (160) 

B 7re 2 ln[cosh(7r/ v /o : )] m^oo U(p[m)}\ l l m 

We plot our numerical findings in Figure [3} The specific values reported in Table[T]are visible 
at the integer values a = 1,2,3,4,6. It is interesting that the coding efficiency figure 0.171 
reported for a = 1 seems to be the global maximum value achievable by our construction 
over all values of a. This is possibly the best performance achievable by minimally supported 



30 



0.25 



sf 0.15 




0.05 



Figure 3: Numerical computation of the coding efficiency formula given by (160) for the 
minimal subordinate filters n^. 



filters that are constrained by subordinacy. We do not know if this figure can be exceeded 
without the subordinacy constraint. The case of arbitrary optimal filters (i.e., not necessarily 
minimally supported) is also open, along with the case of quantization rules that are more 
general than the greedy rule. 
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A Some useful properties of Chebyshev polynomials 



Recall that the Chebyshev Polynomials of the first and second kind in x = cos 6 are given 
by 

T m (x) = cosmO and U m (x) = — — — ^ — — , (161) 

smu 

respectively. The Chebyshev polynomials have, in particular, the following properties (see 
M, 0): 

• T m( x ) = mZ7 m _i(x), 

• The zeros of £7 m _i are Zj = cos {^E 1 ^) , j — 1, . . . , m — 1, 

• For m > 0, the leading coefficient of T m is 2 m ~ 1 , 

• The Chebyshev polynomials satisfy the following identities 

sinh(mT) 

T m (coshr) = cosh(mr), (7 m (coshr) = — — , (162) 

• The Chebyshev polynomials satisfy the differential equation 

(1 - xfT'^x) - xT' m {x) + m 2 T m {x) = 0. (163) 

We say that a polynomial p of degree m has the equi- oscillation property on [—1, 1] 
(compare [3]) if it has m — 1 real critical points (i, . . . , ( m -i which satisfy 

Co := -1 < Ci < • ■ • < Cm-i < Cm := 1 (164) 
such that the associated values are alternating 

P(0) = (-l) m " J (165) 

for j = 0, . . . , m. 

Note that if a polynomial has the equi-oscillation property then its leading coefficient is 
positive. The Chebyshev polynomials of the first kind T m have the equi-oscillation property 
for all m. Indeed, the first two properties given above imply that the Zj's are the critical 
points of T m , and a simple calculation shows that T m (zj) = (— l)" 1- - 7 . The equi-oscillation 
property in fact characterizes the Chebyshev polynomials of the first kind: 

Proposition A.l. If p(s) is a polynomial of degree m in s with the equi-oscillation property 
on [—1, 1], then p = T m . 



32 



Proof. The proof follows ideas used in [3] to establish that, up to a constant, the T m are the 
unique monic polynomials with minimal L°° norm. 

Let p(s) = a p s m + . . . and q(s) = a q s m + . . . be two polynomials with the equi-oscillation 
property. W.l.o.g. assume a q > a p > 0. Let d < • • • < Cm-i be the critical points of p in 
[-1, 1] and set Co = -1, Cm = 1. 

Consider the polynomial r(s) = p(s) — ^q{s) of degree (m — 1). Then r(( m _j) > for 
all even j, and r{C, m -j) < for all odd j. The proof that r = follows from the following 
more general statement: 

Claim: If t < t\ < ■ ■ ■ < t rn e R and a polynomial p of degree m — 1 satisfies 
(-l)V(Cj) > for all j, then p = 0. 

We conclude that r = p — ^-q = by applying the claim to p = (— l) m r. Since p(l) = 
q(l) = 1 implies that a p = a q , we see that p = q. □ 

Proof of Claim: The proof proceeds by induction in m. In the case m—1, p(to) > 
and p{t\) < implies that r = 0. For the induction step, assume that the claim holds true 
for m. Given a polynomial p of degree m with the property, it must have a zero z with 
t m < z < t m+ i. Define p(x) = ^3^. Note that p is a polynomial of degree m — 1. 
then p(tj)(—iy > for < j < m and hence p = by the induction hypothesis. If z = t m 
then p(tj)(— iy > for < j < m — 1, but clearly one also has p(t m+ i)(— l) m > 0. Again 
by the induction hypothesis p = 0. □ 

The following lemma plays a useful role in solving the relaxed minimization problem. 

Lemma A. 2. Let Zj, j — 1, . . . , m — 1, be the critical points of the Chebyshev polynomial of 
the first kind T m , as above, and set zo = —1. Then 




for k > 



for k = 



(166) 



Proof. Recall that T m has leading coefficient 2 m . We obtain 



m—1 



T;(^)=m2 m - i n(^-^) 



(167) 



i=l 



and 



m—1 m—1 




(168) 
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and hence for 1 < k < m — 1 



m— 1 



T'^z k ) = m2 m - 1 H(z k - Zi ) 



i=l 



Zi). 



(169) 



(170) 



i=0 



Thus for 1 < k < m — 1 one has T' m (z k ) = and T m (z k ) = (— l) m , and so (163) reads 



m2 



rn-l m - 1 



i - zi)'^— n ( Zk - Zi ) + m \-ir- k = o, 



i=0 



or 



m— 1 



J3 (z* - «i) 



i=0 



(_l)™-k-l m 

2— !(1-^)' 



(171) 



(172) 



On the other hand, as Zq = cos(7r), we have using (161) 
- *) = ^TT r m (^o) = ^n^m(^o) = ^ lim 

1=1 



2™- 1 o^w sin6 



-l) m - l m 

Om-l 



;i73) 



□ 



References 

[1] J. J. Benedetto, A. M. Powell, and O. Yilmaz. Sigma-Delta (SA) quantization and 
finite frames. IEEE Trans. Inform. Theory, 52 (5): 1990-2005, 2006. 

[2] B. Bodmann and V. I. Paulsen. Frame paths and error bounds for sigma-delta quanti- 
zation. Appl. Comput. Harmon. Anal., 22, 2007. 

[3] P. Borwein and T. Erdelyi. Polynomials and polynomial inequalities, volume 161 of 
Graduate Texts in Mathematics. Springer- Verlag, New York, 1995. 

[4] A. R. Calderbank and I. Daubechies. The pros and cons of democracy. IEEE Trans. 
Inform. Theory, 48(6):1721-1725, 2002. Special issue on Shannon theory: perspective, 
trends, and applications. 

[5] I. Daubechies and R. DeVore. Reconstructing a bandlimited function from very coarsely 
quantized data: A family of stable sigma-delta modulators of arbitrary order. Ann. of 
Math, 158:679-710, 2003. 



34 



[6] I. Daubechies, R. DeVore, C. S. Giintiirk, and V. A. Vaishampayan. A/D conversion 
with imperfect quantizers. IEEE Trans. Inform. Theory, 52(3):874-885, 2006. 

[7] C. S. Giintiirk. One-bit sigma-delta quantization with exponential accuracy. Comm. 
Pure Appl. Math., 56:1608-1630, 2003. 

[8] C. S. Giintiirk. Approximating a bandlimited function using very coarsely quan- 
tized data: improved error estimates in sigma-delta modulation. J. Amer. Math. Soc., 
17(l):229-242 (electronic), 2004. 

[9] C. S. Giintiirk and N. T. Thao. Ergodic dynamics in sigma-delta quantization: tiling 
invariant sets and spectral analysis of error. Adv. in Appl. Math., 34(3):523-560, 2005. 

[10] H. Inose, Y. Yasuda, and J. Murakami. A telemetering system by code manipulation - 
AS modulation. IRE Trans on Space Electronics and Telemetry, pages 204-209, 1962. 

[11] F. Krahmer. An improved family of exponentially accurate sigma-delta quantization 
schemes. In Wavelets XII Edited by Van De Ville, Dimitri; Goyal, Vivek K.; Papadakis, 
Manos. Proceedings of the SPIE, volume 6701, October 2007. 

[12] S. R. Norsworthy, R. Schreier, and G. C. Temes, editors. Delta-Sigma- Converters: 
Theory, Design and Simulation. Wiley-IEEE, 1996. 

[13] R. Schreier and G. C. Temes. Understanding Delta-Sigma Data Converters. Wiley-IEEE 
Press, 2004. 

[14] G. Szego. Orthogonal Polynomials. Amer. Math. Soc, 1939. 

[15] O. Yilmaz. Stability analysis for several sigma-delta methods of coarse quantization of 
bandlimited functions. Constructive Approximation, 18(4):599-623, 2002. 



35 



